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ABSTRACT 


Crew rescue and equipment retrieval is a Space Station 
Freedom requirement. During Freedom's lifetime, 
there is a high probability that a number of objects will 
accidentally become separated. Members of the crew, 
replacement units, and key tools are examples. 
Retrieval of these objects within a short time is 
essential. 

Systems engineering studies were conducted to identify 
system requirements and candidate approaches. . One 
such approach, based on a voice-supervised, intelligent, 
free-flying robot was selected for further analysis. A 
ground-based technology demonstration, now in its 
second phase, was designed to provide an integrated 
robotic hardware and software testbed supporting 
design of a space-borne system. 

The ground system, known as the EVA Retriever, is 
examining the problem of autonomously planning and 
executing a target rendezvous, grapple, and return to 
base while avoiding stationary and moving obstacles. 
The current prototype is an anthropomorphic 
manipulator unit with dexterous arms and hands 
attached to a robot body and latched in a manned 
maneuvering unit. A precision air-bearing floor is used 
to simulate space. Sensor data include two vision 
systems and force/proximity/tactile sensors on the 
hands and arms. 

Planning for a shuttle flight experiment is underway. A 
set of scenarios and strawman requirements were 
defined to support conceptual development. Initial 
design activities are expected to begin in late 1989 with 
the flight occurring in 1994. The flight hardware and 
software will be based on lessons learned from both the 
ground prototype and computer simulations. 

Presented at Space Operations Automation and 
Robotics Conference and Workshop, JSC, Houston, 
TX, 77058, July 25-27, 1989. 


INTRODUCTION 


A requirement exists to provide a retrieval capability 
for objects (astronauts, equipment, and tools) which 
have separated from Space Station Freedom. An 
analysis of the amount of crew Extra Vehicular Activity 
(EVA) likely during the lifetime the Space Station 
indicates, with high probability, that a number of 
objects will accidentally become untethered. Crew 
safety is top priority. In addition equipment may be 
too valuable to lose because it is required in operations 
and replacement is not available on the station. There 
is also collision potential on later orbits which, though 
small, has occurred previously. 

The Space Station itself will lack the capability to chase 
separated crew or equipment and other vehicles such as 
the Space Shuttle orbiter or the Orbital Maneuvering 
Vehicle will not usually be available. Potential solutions 
based on manned, teleoperated, and autonomous 
capabilities have all been proposed. 

Retrieval by a crew member using a Manned 
Maneuvering Unit (MMU) was examined in some 
detail. Analysis revealed that a short response time is 
critical. Many hours of real-time simulation of 
retrievals indicated that manned retrievals were 
unlikely to provide the required response time. In any 
case a major and unacceptable risk to the astronaut 
was involved. 

The evolving requirements call for an unassisted 
deployment from a mounting on the external part of 
the airlock with propulsion capabilities provided by a 
more power full version of the existing MMU. 
Performance guidelines include target retrieval within 
120 minutes of subsystem deployment, Reiiablity 
considerations mandate the use of fault tolerant and 
fail-safe designs with embedded fault detection and 
isolation capabilities. Safety, reliability, robustness, 
and maintainability in space are key attributes. 
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Space Station Freedom advanced automation and 
robotics has been the subject of numerous symposia 
and papers [1, 2], Appropriate roles for humans and 
machines in an evolving mix have been highlighted as a 
specific goal, with supervised intelligent system designs 
as ways to meet the needs of appropriate flexibie- 
capability automation and robotics, thereby giving 
people-amplifier-type productivity gains. 

The retrieval problem provides an opportunity to 
evaluate such systems in the form of a supervised, 
intelligent, free-flying space robot. The concept of 
supervised, intelligent, autonomous robotics provides 
for autonomous behavior of an intelligent type where 
human control is normally at a high level of goal-setting 
and involved in mixed initiative communication as a 
means of implementing decentralized, delegated 
management. By contrast, telerobotics provides a 
partially automated remote extension of human task 
performance with occasional control delegation for 
specific parts of tasks given to the telerobot for 
efficiency reasons. 

Several previous efforts have laid a foundation for 
autonomous robot development including Shakey [3], 
JASON [4], the RPI Rover [5], the JPL Rover [6], and 
the Stanford Cart [7], among others. These first- 
generation autonomous robots were used to explore 
basic issues in vision, planning, and control. However, 
they were all seriously hampered by primitive sensing 
and computing hardware. 

More recent efforts have overcome many of these 
limitations, and very sophisticated second generation 
autonomous robot testbeds have evolved. Some of these 
efforts include the developments of HILARE [8], the 
FMC Autonomous Vehicle [9], the Autonomous Land 
Vehicle (ALV) [10], the various CMU mobile robots [7], 
and the Ground Surveillance Robot (GSR) [11]. A 
more general and complete discussion of autonomous 
vehicle history and technical issues has been given by 
Harmon [12], While operational versions don't exist, 
much advantage can be obtained from these efforts. 

By comparison, the space retrieval task seems simpler 
in some respects. While automatic control, such as is 
available in automatic guided vehicles (AGV), remotely 
piloted vehicles (RPV), and missiles, is not adequate 
here due to the dynamic environment, the more general 
solutions to vision and planning in completely unknown 
environments are not required. There are few objects 
in space; these are cooperative, and largely knowable. 
In low earth orbit, space is characterized by high 
thermal gradients, radiation levels, high vacuum, 
microgravity and reaction-force aspects, and 

constrained and delayed access to information, 
resources, and equipment. Supervision by voice is a 
natural, flexible means of providing the primary 
human-machine interface (supplemented with helmet 
displays) required. This requires limited natural 


language understanding integrated with the 
environment and task as well as functions like planning 
and reasoning. Complete intelligent autonomy of the 
R2D2/C3P0-type is not required nor achievable. 

However, significant technology advances will be 
necessary before even this simple, crucial application 
can be practically addressed. These advances will only 
be gained by implementing autonomous robot 
simulations and testbeds so as to gain experience with 
the developing technology. 

The potential evolution of such a robot to an EVA crew 
helper is obvious. Routine inspections, fetching tools, 
holding objects, could all improve EVA safety and 
productivity. 

The EVA Retriever ground-based technology 
demonstration study [13, 14] was established to design, 
develop, and demonstrate in three phases an integrated 
robotic hardware/software system which supports 
design studies of a space borne crew rescue and 
equipment retrieval capability. Goals for each phase 
were established [15] in support of the overall goal of 
building and evaluating the capability to retrieve 
objects (astronauts, equipment, and tools) which have 
accidentally separated from their spacecraft. The 
Phase I goals were to design, build, and test a retriever 
system testbed by demonstrating supervised retrieval of 
a fixed target. Phase n goals are to initiate simulations 
and to enhance the testbed subsystems with significant 
intelligent capability by demonstrating target retrieval 
while avoiding fixed, arbitrarily oriented obstacles. 
Phase m goals are to more fully achieve supervised, 
intelligent, autonomous behavior by demonstrating 
retrieval of a moving target while avoiding moving 
obstacles. 

Space Station scenarios [16] were examined in some 
detail to aid in the definition of a set of design reference 
missions. A number of systems engineering studies 
were conducted In support of the software design. 
Level A requirements for a projected Space Station 
version were developed in a conceptual design study 
[17]. Level B software requirements were derived in 
greater detail for this possible future Space Station 
application [18]. 

This paper gives an overview of the experimental 
hardware and software and a brief summary of the 
Phase n experiment. These are related to the current 
planning for a shuttle flight experiment. 


PHASE n PROTOTYPE 


The technology demonstrations are being conducted on 
the JSC Precision Air Bearing Floor (PABF). The 
retriever/MMU unit is mounted on a test stand with 
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compressed air supplied thru an umbilical. The MMU 
has twenty-four thrusters, four on each rectangular 
side. The MMU accepts simple translation and rotation 
on/off commands to fire thrusters providing fixed 
acceleration in any of the three translational or three 
rotational directions. 

The current prototype (Figure 1) is an 
anthropomorphic manipulator unit with dexterous 
arms and hands attached to a robot body and latched in 
an MMU. Sensor data include accelerometers, 
gyroscopes, two independent vision systems, and 
force/proximity sensors on the hands and arms. The 
primary vision system consists of a laser scanner imager 
and video camera mounted on a controllable turntable. 
The secondary vision system is a multicamera video 
tracking system, with a chest camera array and a 
camera in one of the hands. 



Figure 1. Retriever test article. 

The prototype has dual 6-degree-of-freedom arms. The 
arms have roll and pitch at the shoulder, elbow, and 
wrist. One of the arms has a dexterous grasping hand 
and the other has a three fingered gripper. The 
dexterous hand incorporates proximity sensors in order 
to support adaptive grasping of an object by 
monitoring force and moment buildup. 

The processor configuration contains seventeen 
transputers (five of which are dedicated to vision 
processing), several 68020 processors, a 80386 
processor, and a special purpose video tracker 
subsystem. 

The EVA Retriever software is required to 
autonomously plan and execute a target rendezvous, 
grapple, and return to base while avoiding stationary 
and moving obstacles. The system is required . to 
monitor plan execution, estimate probability of mission 
success, and dynamically replan whenever needed to 
achieve system goals. 


The software architecture (Figure 2) incorporates a 
hierarchical decomposition of the control system that is 
horizontally partitioned into five major functional 
subsystems: perception, world model, reasoning, 

sensing, and acting. The design utilizes hierarchical 
flow of command and status messages but allows 
horizontal flow of data between components at the 
same level. Computation is performed at the lowest 
possible level and, in general, knowledge-based systems 
are utilizated only when algorithmic solutions are 
lacking In power or flexibility. This approach handles 
multiple levels of abstraction well and permits the 
incorporation of special data paths between time 
critical components. 



Figure 2. EVA Retriever software components. 

The overall design provides for an evolutionary system 
improving in capability over time and as it earns crew 
trust through reliable operation. Additional details on 
the hardware and software design may be found in 
Grickson et. al. [14]. 


PRELIMINARY PHASE II RESULTS 


At the time of this writing, hardware and software 
integration Is nearing completion in preparation for 
PABF evaluations and demonstrations in the summer 
of 1989. Nevertheless, some preliminary results are 
available. Several static tests have been successfully 
completed which indicate successful integration and 
operation of subsystems. 

The computer simulation testing carried out in Phase II 
for purposes of unit and integrated software dynamic 
testing of prototype designs has already paid major 
dividends. First, we have gained solid confidence in the 
Retriever's behavior and the software which drives this 
behavior. Second, we have been able to investigate 
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detailed requirements issues and design issues into 
which we otherwise would have little means to gain 
insight. Third, as a result, we have been able to make 
numerous minor requirements, design, and 
implementation changes and test them before hardware 
integration or PABF evaluations. So, as expected, the 
simulation testing has been very useful and efficient 
even though it is not the whole answer. Building the 
complete artifact and testing it in a set of physical tests 
including grappling cannot be simulated and is 
required to gain confidence in the design and 
understand its limitations. 

In attempting to design realtime visual perception for 
grasping some preliminary results were obtained which 
relate to sensors and engineering of computer hardware 
and software for robotic applications. Images of 
moving objects taken with the laser scanner at 0,8 
seconds per frame show a warped and distorted image 
of the objects for even reasonably slow moving objects. 
This result indicates that frame rates need to be 
increased substantially to deal with moving objects. 

The range image processing from laser scanners done 
by Vemuri and Aggarwal (19) using curvature 
representations in Constructive Solid Geometry (CSG) 
approaches for stationary objects establishes, in 
principle, an algorithm for constructing the surface of 
an arbitrary, 3-D unknown man-made object and, in 
particular, demonstrates the need for spline fitting, 
which is computationally intensive. Even though this 
algorithm is parallel in each "patch" of the image, 
extrapolation from the Vax sequential implementation 
to a parallel transputer one still leaves a computational 
period of many minutes which is not near enough to 
realtime to be a practical solution for robotic grasping. 
A simpler representation with less computational 
requirements seems indicated and is being sought 
without loss of generality. There are also sensor 
requirements implied by this algorithm which means 
many more pixels from the object are needed than our 
current scanner provides (that is, the IFOV must 
subtend a much smaller angle which coupled with 
higher frame rates may require greater laser power). 

The performance measurement and debugging function 
is difficult conceptually because defining good measures 
of performance is not easy, but some progress has 
already been made. The motivation is simple - with 
measurements of performance occurring constantly, the 
Retriever, its supervisor, and its designers can know 
how it is performing. As we are focused at the moment 
on requirements and design, measurements of 
performance provide the data needed to quantify 
limitations and thus where design improvements will be 
most useful. One measure of performance is the time 
required for rendezvous for a given arrangement of 
obstacles and target. Another is the thruster durations 
during this rendezvous needed for translations and 
rotations, A third is the distance travelled. In space, 


minimum time and minimum fuel trajectories are of 
interest. On the PABF, "good" trajectories weigh 
distance, time, and fuel. 

Preliminary measurements of time for rendezvous and 
total thruster durations for one distance and 
object/target scenario from testing against simulation 
gave about 130 seconds for rendezvous and 13 seconds 
total thruster duration for the case with translations 
and rotations (including the head) occurring in parallel. 
Measurements on the air bearing floor will be 
compared to these simulation results. 

Debugging is another important operational function. 
Although we have used the best third-party debugging 
software we could find for a multi-processor transputer 
configuration, we do not yet have an adequate 
capability, meaning it takes too long to find bugs 
because the tools are not supportive enough. 

Another result from simulation testing dealt with minor 
modifications to the design of the world model in 
reasoning about whether an object was an object seen 
before or a new object. Thi s de als with robustness to 
inertial measurement drift in space when no spacecraft 
radar is available for tracking Retriever and multiple 
objects. The event can be described as an unsensed 
change of location of Retriever. On the PABF this can 
occur due to floor disturbances causing sliding not 
sensed by accelerometers or gyroscopes. We use vision 
derived object location to provide feedback to Retriever 
about Retriever location. However, when an object 
appears to be in a relative location where no object was 
seen the last look and far enough from where one was 
seen, we call it a new object. Our design is now more 
robust in this one respect due to simulation testing and 
subsequent design modifications. 

Another testing/modification case of this same kind 
dealt with parallel actions, that's doing two or more 
things simultaneously rather than sequentially. In the 
situation where a new object is seen while on a planned 
rendezvous trajectory, motion continues until a new 
motion plan is computed, or if too close to impact, a 
hold is executed until a new plan is available. We 
expect reactive planning to further improve our 
responsiveness. 


DYNAMIC ENVIRONMENT 


Phase III of the Ground Demonstration Program was 
intended to deal with moving objects in a dynamic 
environment (Phase II dealt with stationary objects). 
Consequently, requirements, design, implementation, 
and evaluation of techniques for moving objects have 
been planned for 1990. 


562 


An overview of our technical approach to moving 
objects is simply that Retriever needs to formulate and 
execute plans using visual perception for object search, 
acquisition, recognition, tracking, and grappling. 
Retriever will formulate and execute plans for mobility 
with moving obstacle avoidance to rendezvous with the 
moving target. Retriever will use dexterous 
manipulation and a grappling mechanism for 
grappling, tethering, and transfer. The plans need to 
be adaptable to the specific situation and to compensate 
for unknowns with reactive plans which can tolerate 
failed actions or react to unexpected events. 


Safety Policy in Software 


A dynamic environment raises the issue of providing 
safety. Retriever must be safe to use as it carries out its 
retrieval tasks, which will vary, in the face of 
unintended contacts, failed actions (grapples), and 
mechanical failures. We mean by being safe that 

Retriever must not harm an astronaut, any part of 
another spacecraft, or itself. We will employ 
technology (to provide safety) which supports 
guarantees on robot behavior. 

Our approach is to provide safety via two approaches. 
The first is software safety technology which is 
concerned with ensuring that software will execute in a 
systems context without resulting in states of 
unacceptable risk and will take actions to remove the 
Retriever from conditions of unacceptable risk if they 
should occur due to detectable hardware or software 
faults or command errors -- a policy that states that 
Retriever software will neither create nor ignore states 
of unacceptable risk. Risk is defined as danger times 
hazard severity where danger is the probability of a 
hazardous state leading to an accident and severity is 
the worst possible damage that could result. A hazards 
analysis for software is planned for 1990. 

The second approach is to provide safety via reactivity 
where we will encode plans as networks of responses to 
possible situations and where guarantees on behavior 
are sought via a finite state machine analogy. 


SHUTTLE FLIGHT EXPERIMENT PLANNING 


Planning for a Shuttle flight experiment in 1994 is 
underway. The primary technical objective of the flight 
experiment is to develop and demonstrate from the STS 
base a flight prototype of a crew supervised, 
intelligently autonomous space robot for the retrieval of 
free-floating objects. A tentative schedule of activities 
required is shown in Figure 3. 
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Figure 3. Preliminary Flight Experiment Schedule. 

A preliminary set of scenarios were defined to support 
conceptual development. The primary demonstrations 
are the autonomous robotic retrieval of free floating 
targets in space (tool, astronaut model). The set of 
scenarios is organized in order from simple to more 
difficult to maximize success and minimize risks. There 
are essentially three groups of scenarios: those in the 
payload bay prior to commitment of EVA; those 
acquisition, tracking, and grappling tests in the payload 
bay with an EVA astronaut as supervisor; and free- 
floating target retrieval demonstrations well out of the 
Shuttle bay organized in order of increasing difficulty. 

These latter demonstrations include a fixed target 
retrieval held by the Remote Manipulator System on an 
extension pole, a free- floating but stable tool retrieval, 
a rotating tool to be retrieved using a grappling 
mechanism (such as a net) , a tool retrieval with 
obstacle avoidance (no obscuration of target), a tool 
retrieval with temporary obscuration of the target by 
an obstacle, and an astronaut model retrieval. 

Initial requirements and design activities are planned to 
begin late in 1989 for the actual flight in 1994. The 
necessary evaluations of approaches to deal with 
moving objects in a dynamic environment must 
factored into the design. In fact, flight hardware and 
software will be based on lessons learned from both the 
ground prototype and a complementary series of 
computer simulations of moving objects. Flight system 
reliability will be better understood, crew training will 
be easier, and Shuttle integration will be less costly due 
to the use of the previously flown MMU. 

The Crew and Equipment Retrieval Systems (CERS) is 
an identified part of the Space Station. A CERS study 
is currently underway by the Space Station Work 
Package 2 contractor, McDonnell Douglas. The study ls 
currently in Part I which is a requirements definition 
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phase. Part I products will include a description of 
concepts evaluated and their programmatic impacts, 
and recommendation for retrieval capability 
implementation for both Station PMC and for Station 
Phase II. The Part I study will also define a study plan 
for the Part II CERS study. The Part II study would be 
utilized to produce at least two point design concepts. 
One concept would possibly be a simple astronaut self- 
help device for use by a conscious crew member in the 
early Station operation. The second design concept 
produced would be a free-flying retrieval system 
capable of satisfying all appropriate crew/equipment 
retrieval requirements. 

The CERS study is directed to consider a supervised 
intelligent autonomous system such as the EVA 
Retriever and its related test programs. The EVA 
Retriever flight experiment would thus serve as a test 
bed for a variety of functions and equipment directly 
applicable to the Space Station CERS. This includes 
tracking, control, target recognition, grapple methods, 
crew control, and Station interface definition. 


CONCLUSION 


Evaluation of improved technology for the practical 
realization of a potential solution to the need for 
retrieval of crew and equipment in space near their 
spacecraft is underway. Preliminary results from the 
second phase of the ground testbed activity have been 
obtained from both computer simulations and the 
ground prototype. Important evaluations of technology 
to deal with a dynamic environment of moving objects 
including safety software are planned for 1990. 
Hardware and software lessons learned wiTl be factored 
into planning for a Shuttle flight experiment. 
Assessment of practicality will rest on experimental 
evidence when these are completed. 
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